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ABSTRACT 



Lehman College of the City University of New York has begin 
a 5-year project to improve the retention of first- time, full-time regularly 
admitted freshmen through the implementation of a student development model. 
The development of this model is described and the results of some 
preliminary studies are presented. As part of the project, researchers are 
testing the seven-stage Kreuzer- Johnson (P. Kreuzer and A. Johnson) Student 
Development Model and investigating whether a set of defined measures can be 
used to determine passage through the seven stages of development and to 
predict attrition or retention. This study focuses on the reliability of the 
first part of the Student Experience Survey, which will measure attitudes 
toward the college experience. This instrument is a 50-item questionnaire 
with Likert scale items that can be divided into 4 10-item subscales to 
assess attitudes toward specific aspects of the student experience. The pilot 
of the questionnaire was administered to 340 freshmen in a Freshman Year 
Initiative program, and the reliability of the total test was found to be 
0.81. Item analysis identified 4 flawed items, and the revised survey was 
administered to 284 students, yielding a total test reliability of 0.89. The 
next step will be to determine if the survey had predictive validity. (SLD) 
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IMPROVING RETENTION THROUGH A COMPREHENSIVE STUDENT 

DEVELOPMENT MODEL 
Anne D. Johnson and Paul G. Kreuzer 
American Educational Research Association Annual Meeting 

April 12, 2001 
Seattle, Washington 



Introduction 

Like many institutions of higher learning across the coimtry, and especially those that 
serve a non-traditional student population, Lehman College of the City University of 
New York is faced with a high attrition rate and lower than desirable graduation rate after 
six years. With support from the United States Department of Education in a Title V 
grant for Strengthening Institutions-Hispanic Serving Institutions, the College has begun 
a five-year project designed to improve our retention of first-time, full-time, regularly 
admitted freshmen through the implementation of a student development model. 



As part of this project, we are testing the seven-stage Kreuzer-Johnson Student 
Development Model based on the life span, professional/vocational development theory 
proposed by Donald H. Blocher and Rita S. Rapoza. (Arthur W. Chickering and 
Associates, eds.. The Modern American College, 1981) The overall objective of our 
research is to determine if a set of defined measures including academic preparation 
measures from high school, academic achievement measures from college, and attitudinal 
measures from college can be used to determine passage through the seven stages of 
development and to predict attrition or retention. Establishing valid predictors will 
enable us to provide strategic interventions to help students stay in school. 

Although academic preparation and achievement variables are well-established predictors 
of success, we believe that student attitudes toward the college experience may also play 
a significant role in determining whether an individual will persist and graduate. 
Currently, we are in the process of testing the first of two survey instruments. Parts I and 
n of the Student Experience Survey. Using Part I of the survey, we are measuring a set 
of attitudes toward the college experience among a pilot cohort of students that entered 
Lehman in fall 1 999. Our current research objective is to establish the reliability of Part I 
of the Student Experience Survey. 

Background 

Establishing improved retention of first-time, fijll-time freshmen as a college goal 
provided the impetus for a study to understand why students persist. An analysis of 
persistence data, a student questionnaire, and anecdotal evidence of student conversations 
wth feculty, professional advisors, career counselors, and a part-time personal counselor 
in the College’s Freshmen Year Initiative program revealed that although many students 
leave for reasons the College cannot address — ^financial and family problems, for 



example— one of the greatest problems facing our students is a lack of acculturation to the 
academy. Based on many decades of experience in teaching, advising, and counseling 
students, we also believe that the acculturation process, culminating in academic success 
and vocational fulfillment, is a developmental process. 

Thus, we considered a number of theories, such as those proposed by Loevinger and 
Weathersby; Perry; Kohlberg and Gilligan; and Tolbert (see summary in Chickering, 
1981, pp. 774-775), and chose the professional/vocational theory of life span 
development suggested by Blocher and Rapoza as the basis for a student development 
model that would reflect the particular growth pattern we believe our successful students 
e^qjerience. 

The Kreuzer- Johnson Student Development Model has seven stages: 

I. Applying, 

II. Enrolling, 

III. Succeeding vsdth Courses, 

IV. Attaining Overall Academic Success, 

V. Progressing toward Career Goals, 

VI. Moving toward Independence, and 

VII. Defining Self in Terms of Education and Profession. 

Each stage represents a moment in an overall continuum of a student’s relationship with 
the College, as well as his or her intellectual development, progress toward a fulfilling 
career, and understanding and appreciation of the complex interrelationship between 
academic development and professional/vocational success. 

Most of our students apply to college because their families, teachers, and society in 
general expect them to do so. They also may apply because they have grown up hearing 
the cliche, “to get a good job, get a good education,” and because going to college solves 
the problem of what to do immediately following high school. Most entering students, 
however, are naive about what the College has to offer and the demands of college-level 
work. Although many can articulate their career aspirations, they may not fully 
understand the variety of skills needed for their chosen profession or know of the 
multitude of possibilities open to them. Because many families place unreasonable 
responsibilities on the students, learning coping strategies and time management skills 
becomes an important part of the early college experience. In addition, many students 
must learn to balance their motivation for attending college — long-term financial and 
professional success — ^with their realization that attending college may delay short-term 
desires for independence fi-om family and financial gain. For students to succeed, we 
have to counter many of the beliefe with which they arrive and help them progress 
through the stages of the Student Development Model toward a more sophisticated and 
realistic set of expectations fi’om the college experience. 

To establish a means for determining where on the continuum an individual might be and 
thus help students progress, we began to define a set of measures to use as criteria for 
determining student passage through each stage. Recognizing that academic preparation 



in high school is always significant in long-term student success, we knew we would 
have to consider such variables as number of academic units fi-om high school, high 
school grade average, placement test scores, and SAT scores. We also knew we would 
have to consider standard achievement measures at the College such as GPA, declaration 
of major, and number of credits earned over time. 

In order to find out how students feel about their progress or whether they are developing 
more realistic ideas about their experience, we developed a two-part survey instrument; 

• Student Experience Survey Part I, which is to be administered in the first, second 
and fourth semesters, includes a set of scales for measuring the students’ attitudes 
toward their 

1 . Use of Basic Study Skills, 

2. Satisfaction with Academic Progress, 

3. Satisfaction with the Career Orientation of Courses, and 

4. Social Integration. 

• Student Experience Survey Part II, to be administered in the sixth and eighth 
semesters, includes a set of scales for measuring the students’ attitudes toward 
their 

1 . Satisfaction with Academic Success, 

2. Satisfiiction with Progress toward Establishing a Career, 

3. Use of Advanced Cognitive Skills, and 

4. Satisfaction with Emerging Independence. 

• In addition to these attitudinal subscales, we also developed a set of questions 
included on both parts relating to thoughts about 

1. Leaving School, 

2. Relationships with Faculty or Professional Staff, 

3. Satisfaction with Student Services, and 

4. Overall Feelings about Belonging at the College. 

We are currently testing the reliability of Part I of the Student Experience Survey. 
Limitations and Assumptions 

There are two major limitations to this research. Although a long-term goal of the project 
is to establish the validity of the instnunent, currently we can only claim a crude form of 
fece validity. Having established a set of attitudes to associate with each stage of 
development, we have developed survey questions that we believe, and which appear on 
the surfece, to relate to those attitudes. 

A second limitation is that we are working with a specific student population. Lehman 
College, a public conq)rehensive institution offering both baccalaureate and masters 
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programs, is located in the Bronx, the poorest borough of New York City. The College 
serves an urban, generally poor, largely minority, and first-generation college student 
population. Many students speak a language other than English at home, and all students 
commute to the College. Students who begin as full-time fi-eshmen are generally of 
traditional college student age, whereas the students who transfer to the College or who 
begin part-time are generally working adults with children. The assumptions that 
underlie this project are based on our experience with first-time fi'eshmen students. 
Although we believe that the model will be widely applicable, establishing its predictive 
power at Lehman does not necessarily ensure that it will work with different types of 
students at different types of institutions. 

In designing the project, we have made a number of basic assumptions. The most 
overarching is that a model of student development can be used to predict retention or 
attrition, and that such a model can help the College develop appropriate intervention 
strategies to increase the percentage of students retained through graduation. A related 
assumption is that the profile of our student population suggests a set of high-risk 
variables, such as socio-economic status and familial lack of experience with the 
institutions and expectations of higher education, that lead to attrition. We also believe 
that attitudes toward academic and career progress affect a student’s commitment to 
staying in school. In addition, we believe that one of the major factors that result in 
student attrition is that many first-generation students enroll with the expectation that the 
college experience will be entirely vocational or career preparatory in nature. These 
students may be finstrated by an inability to connect their academic work to the career 
preparation they expect. Similarly, they may not imderstand or recognize the academic 
values of philosophical inquiry, critical thinking, and development of higher order 
cognitive skills that underlie many faculty members’ approach to teaching. Thus, we 
assume that variables other than those associated with academic preparation, which are 
known to be strong predictors of success, may be related to retention. In particular we 
believe that student attitudes toward a variety of aspects of the college experience are 
important factors in predicting retention or attrition. 

In designing our survey instrument, we have assumed that we can use Likert-scales to 
measure student attitudes toward definable aspects of the college experience and that we 
can link these scales to the particular stages of the hypothesized student development 
model From our own experiences with students and fi-om the standard literature on 
retention, we believe that the variables we have included in the Kreuzer-Johnson Student 
Development Model (e.g., contact with faculty and professional staff, finding fiiends on 
campus, establishing mentoring relationships with people in a chosen career path, and 
registering for internships or project courses) as well as more standard measures (e.g., 
grade point average, success in completing credits, high school units completed, high 
school average, and standardized test scores) are useful predictors of attrition or retention. 

The Student Experience Surveys have been constructed to help assess student progress 
through this model. Both consist of two 50-item questionnaires each with: 

• 40 Lflcert-scale items that are dividable into 4 ten-item subscales to assess 
attitudes toward specific aspects of the student’s college experience; 
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• 3 items related to student satisfaction with academic advising, career counse ling, 
and personal counseling; and 

• 7 items related to campus afiSliations. 

For each of the attitude subscales, we have written five sets of paired items. Each pair 
consists of one positive and one negative statement. For example, a pair of statements in 
the Survey designed as part of the subscale on attitude toward academic progress is: 

• I have enjoyed my course work this semester. 

• My course work this semester has been unpleasant. 

Students indicate their level of agreement with the statement on a scale of 1 to 5, fi-om 
strongly agree to strongly disagree. The items are scored fi^om one to five with reverse 
scoring on the negative items. With a score of 3 being neutral and an understanding that 
the real limits of the number extend fi-om 2.5 to 3.5, we have assumed that a score of 3.5 
on a particular item or of 35 on a subscale that includes 10 items indicates a positive 
student attitude toward the variable in question and can be used to help measure student 
passage through the associated stage of development. 

Methodology 

We are using fi’eshmen who began in fall 1999 as the cohort to follow longitudinally 
through all phases of developing the model. Part I of the Survey has now been 
administered three times to students in either in fi’eshman year orientation sections or 
fi'eshman English classes. The survey was first piloted in the Fall semester of 1999. A 
revised version was given in the Spring of 2000, and both new fi-eshmen and second 
semester students were invited to take it again in Fall 2000. Using Cronbach-alpha 
correlation, we have determined the overall reliability of Part I of the Student Experience 
Survey, and using item analysis, we have determined the reliability of the subscales. 

Research Findings to Date 

We administered the pilot of Part I of the Student Experience Survey to 340 students in 
Freshman Year Initiative classes in Fall 1999. Using the Cronbach-alpha correlation 
coeflScient, we found: 

• the reliability of the total test to be .81 

Using item analysis we found the reliability coefficients for the subscales as follows: 

• Attitude toward Basic Skills = .75 

• Attitude toward Academic Progress = .76 

• Satis&ction with Career Orientation of Courses = .75 

• Social Integration = .74 
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The item analysis from the first pilot administration for each subscale indicated a need to 
change one positive/negative pair related to working while in college. The item analysis 
for the fill! 40-item test indicated that the logic of four items on study skills was flawed. 
The items have been revised and the revised survey was administered to 284 students in 
the Spring of 2000. Using the Cronbach-alpha correlation coefficient we found: 

• the reliability of the total test to be .89 

Using item analysis we found the reliability coefficients for the subscales as follows: 

• Attitude toward Basic Skills = .74 

• Attitude toward Academic Progress = .84 

• Satisfaction with Career Orientation of Courses = .86 

• Social Integration = .79 

This second analysis indicated a need to revise two items that were changed before the 
Fall 2000 administration of the survey. These items were revised and the final version of 
the survey was administered to 259 students in the Fall of 2000. Using the Cronbach- 
alpha correlation coefficient we found: 

• the reliability of the total test to be .89 

Using item analysis we found the reliability coefficients for the subscales as follows: 

• Attitude toward Basic Skills = .78 

• Attitude toward Academic Progress = .81 

• Satisfaction with Career Orientation of Courses = .89 

• Social Integration = .74 

Next Steps 

Our next task is to begin to determine if the survey has predictive validity. In order to do 
this, we will correlate the fall 1999 cohort Student Experience Survey data (total score, 
four subscales, and 10 items) with both spring 2000 and fall 2000 enrollment. These 
procedures will indicate the degree to which the instrument alone can predict retention or 
attrition. We are using this preliminary pilot cohort for analysis because we are 
intervening with Fall 2000 cohort students based on attitude measures and other scores. 

We also are currently designing a multiple regression analysis protocol for correlating 
academic preparation measures, academic achievement measures, and Student 
Experience Survey scores with retention and attrition. Even though we continued to 
revise Part I of the Student Experience Survey, we believe it is important to generate its 
preliminary predictive validity measures and to begin to assess the predictive power of 
the Kreuzer- Johnson Student Development Model. 
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While the predictive validity of the instrument is yet to be confirmed on our fall 1999 
entering cohort, we are initiating a number of intervention strategies during the spring 
2001 semester with our Fall 2000 cohort. With a cadre of peer educators, we are calling 
students who score less than 35 on any of the four subscales in Part I of the Survey. 
Those students whose subscale scores indicate concerns about study skills or academic 
progress will be contacted by peers working in the Academic Information and 
Advisement Center. Those whose subscale scores indicate concerns about the career 
orientation of courses will be contacted by peers in the Career Services Center, and those 
whp appear to be concerned about social integration will be contacted by peers in the 
Counseling Center. In all cases, these contacts are designed to encourage students to 
discuss their issues of concern with the College’s professional advisors and counselors. 

In the meantime we are also developing a Student Development Database for storing 
student data that is not captured on the College’s Student Information Management 
System such as referrals of students from faculty and professional stafif, survey data, and 
student contact with advisors, career counselors, tutors, and faculty. Constructed to 
access the standard information from the College system as well as its own data, the new 
system will allow sophisticated analysis of the academic preparation, academic 
achievement, and the attitudinal measures that can be used for identifying students in 
need of interventions related to the stages of the Kreuzer- Johnson Student Development 
Model and for other personalized communications such as letters congratulating students 
on a successful semester or inviting those who seem to be ready for internships courses 
for a targeted workshop. 



Conclusion 

We have now determined that Part I of the Student Experience Survey is statistically 
reliable. Therefore, we are ready to use it widely as a survey instrument and to employ it 
as part of our overall model. With the reliability established, we can begin to determine 
its predictive power both as an instrument by itself and in conjunction with the other 
variables of the Kreuzer- Johnson Student Development Model. 
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